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The primary objective of this study invoivea exploration and 
refinement of moderated scoring- key techniques for scorin^; psychological 
inventories. V/ork centered in two areas: (l) Development of procedures 

for determining the optimum number of subgroups (and hence, moderated 
keys) required for maximizing the predictive effectiveness of an inven- 
tory; and, (2) Development of a single scale for reporting the scores 
obtained from a set of moderated keys. The seconcaxy objective involved 
determination of the accuracy of moderated scoring Key predictions of 
college attendance as compared with predictions obtained from conventional 
keying techniques . 



Basic data consisted of biographical inventory responses and academic 
aptitude test scores for approximately 20,500 high scnool senior boys. 

An equally weighted combination of two academic aptitude tests served as 
the moderator variable. Scorin** ke^/s v-’ere ^onr.eo for each of twelve 
ability level subgroups and various combinations of these groups. Deci- 
sion rules involving comparison of point: biserial correlation coefficients 
for selected keys and key groups were aeviseci in oraer to icientify the 
optimum number of pioderatea scoring keys . Criterion group membership 
probabilities were used to form a coni;ion scale on which to report scores. 

The keying procedures developed as tne primary objective of the projeco 
appear to work well when applied to actual Gama. Large samples will 
generally be required for their use, how^ever . 

Academic ability was not found to be an effective moderator variable 
for the predictor ana criterion unuer study. Althourii a statistically 
significant difference in favor of the renerateu keys over conventional 
keys vras obtained, the practical implications of this aifference were 
slight. 

A hit rate of 11 % was achieved by biograpriical cat a for predictions 
of college attendance and nonattenaance in a crcss-va.l ication saj-ple. 

This rate and the eq-ui valent point biserial correi.atj or. coeificient 
of .60 were substantially higher than the corresnoriaing figures for acaaemic 
aptitude used alone. Thus, biographiccil uata v>ould appear to be a powerful 
predictor of college attendance. 
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Problem 



In the field of measurement, instruments falling in the categoiy 
called tests are characterized as having questions with one and only 
one correct answer. Inventories , on the other hand, contain questions 
which have no^ one correct answer. At best, the "correct" answer to an 
inventory item might be thought of as the response most appropriate to 
the person answering the question. Thus, to the question, "How many 
brothers and sisters do you have?", one person might respond with "one," 
another with "six," and another with "none." 

A test item is easy to score because there is only one correct 
answer. However, no such easy scoring procedure is available for the 
inventory item. Instead, inventory scoring requires the application 
of specialized techniques for developing scoring keys , keying 

techniques). Three general keying techniques are available — logical, 
hcaaogeneous , and empirical techniques. Logical keying techniques date 
back to early, unsophisticated attempts at inventory construction and 
are seldom used by measurement specialists today. Both homogeneous and 
empirical keying methods have unique merits with the latter generally 
seen as yielding more valid predictions of specific criteria. 

In its simplest form, empirical keying involves contrasting the 
inventory item responses of two groups , high school dropouts and 

persisters), identifying those items to which the groups respond dif- 
ferently, and then scoring those items in such a fashion that members 
of one group will tend to receive higher total scores than members of 
the other group. Thus, to the question, "How many brothers and sisters 
do you have?", responses of "two or three" might be scored +1, indicating 
a tendency toward persistence; responses of "six or more" might be scored 
-1; and all other responses may receive no score because they fail to 
differentiate persisters and dropouts. If a sufficient number of items 
are found to contain response options that differentiate between per- 
sisters and dropouts , the inventory can be used with other individuals 
for whom predictions of group membership are desired. 

Numerous refinements of this simple approach have been available 
for some time. For instance, Gleser and DuBois (l 95 l) describe a succes- 
sive approximation method for taking into account both item-criterion 
correlations and interitem correlations. More recently, Ghiselli (i960) 
showed that it is possible to differentiate individuals for whom predic- 
tions have relatively high versus low accuracy by use of an empirically 
keyed moderator variable. Finally, Prediger (1966) demonstrated the 
feasibility of developing separate inventory scoring keys for application 
to individuals scoring at each of several different levels on a logically 
related moderator variable. This approach to empirical keying, known 
as the moderated scoring key approach, appears to represent a new and 
promising method for improving the validity of many types of inventories. 

Surveys of study habits and attitudes, interest inventories, personal- 
ity inventories , and biographical information blanks are among the general 
types of inventories representative of those commonly in use in the nation's 
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schools. In addition to the more coiranon uses, these inventories have 
been applied to the prediction of hiph school dropout, collepe attendance, 
college persistence, and the estimation of delinquency proneness. Yet, 
for many applications, inventories have been found to contribute a dis- 
couragingly small amount of useful information. The development of 
improved keying techniques would appear to be a basic means of increasing 
inventory validity. 

Within this context, the general purposes of this stu dy were: 

(a) To investigate procedures for refining the moderated scoring key 
approach and facilitating its practical application; and (b) to provide 
information on the generalizability and effectiveness of the approach by 
comparing the validity of college attendance predictions based on the 
refined keys with those obtained by conventional procedures. 

As a by-product of this investigation, biographical data related to 
college attendance versus nonattendance was identified for a nationwide 
sample of high school students. Sufficient information is presented in 
Appendices A through C of this report for the development of an effective 
tool for use by school counselors in locating high ability students not 
likely to go to college. 



Related Literature 

Research on the application of moderator variables to prediction 
problems has been reviewed by Dunnette (1963) and Ghiselli (1963). The 
common approach is to develop, by means of empirical keying techniques, 
a scale that differentiates between individuals for whom inventory or 
test predictions have greater and lesser degrees of accuracy. The scale 
is either based on unused items within the predictor itself or on the 
items in a separate measure. The scale so developed is termed a modera- 
tor variable because it is seen as representing a characteristic that 
affects (moderates) or at least reflects different degrees of relation- 
ship between two variables in much the sair;e way that two variables inter- 
act with a third in analysis of variance. These moderator variables 
usually have been found to add little to the accuracy of predictions 
when combined with the predictor in a multiple regression equation. 

Hence, their chief function is to "predict predictability." 



Another application of moderator variables, which in addition to 
facilitating the prediction of predictability , promises aid in under- 
standing the reasons for differential predictability , involves the use 
of logic al rather than empirically derived moderator variables . Thus 
Goodstein and Heilbrun (I9b2), in a study of college sophomores, found 
only one significant relationship between the Edwa rds Per sonal Prefei^^ 
Schedule and GPA when vocabulary level was controlled by means of partial 
correlation. However, when academic aptitude was used as a logical 
moderator variable by dividing both male and female groups into low, 
medium, and high ability subgroups of equal size, a number of statisti- 
cally significant partial correlations appeared, especially for the 
males. In addition, the scales for which correlations were found to 
be significant varied greatly with ability level. 





3 







In a more recent study, Heilbrun (I965), usinf colleg;e persistence 
over a one-year period as the criterion, paired persisters and dropouts 
on the basis of ability and sex. He then divided the male and female 
groups into subgroups of high, medium, and low ability. Hypothesized 
differences between persisters and dropouts on the Adje ctive Che c k L ist 
occurred primarily at the high ability level. Significant ability level 
interaction was found. Hence, in both of the above studies, academic 
ability was used as a logical moderator variable that added to the under- 
standing of the relationship between personality characteristics and 
college success and, at the same time, iaentified subgroups with dif- 
fering predictability. 

It should be noted that Hakel (1966), in a replication of the 
Goodstein and Heilbrun study, failed to substantiate the generality of 
the results. Hakel 's additional finding that none of the EPPS— achieve- 
ment correlations were stable across three hetero gene ous subgroups of 
the total sample may be the result of his small total sample size (11=102). 
The small size of the ability level subgroups could also have a bearing 
on the negative results of his replication. In any case, Hakel justi- 
fiably emphasizes the need for cross-validation and validity generalization 
studies in the investigation of moderator variable effects. 

Among other recent studies involving moderator variables is one by 
Berdie and Hood (1966) in which the relationship between a number of 
measures and the college attendance plans of high school students grouped 
by sex and area of residence was investigated. High school rank, 
scholas-Gic aptitude test scores, personality variables, and biographical 
data were among the predictors used in the study. According to Berdie 
and Hood, the results suggest that predictions of college plans are more 
accurate when prediction equations are developed for homogeneous groups 
of students. In addition, there was evidence of differential predicta- 
bility across the subgroups . It is interesting to note that the modera- 
tor variables in this study were categorical rather than continuous. 

The usefulness of a number of other moderators in specific situations 
has been investigated. For example, Korman (l9o6, 196?) successfully 
used self-esteem as a moderator of the relationship between various self 
perceptions and nature of vocational choice. Clark and Cairipbell (19 5) 
report that ability serves as a moderator variable for the relationship 
between interests and grades. Brown and Scott (1966) were unsuccessful 
in their efforts to improve predictions of grades for various groups of 
college students through the use of study habits, study attitudes, s-^d 
personality variables as moderators. On the other hand, Barger and ha ± 
(1965), using ability as a moderator instead of a predictor , showed 
that parents' marital status, family income, father's education, and 
family size were differentially related to GPA for college females. Ao 
statistically significant relationships were found for males at any of 
the three ability levels under investigation. Parent's marital status^ 
was the only biographical variable found to be related to persistence in 
college. This relationship appeared for both males and females but only 
for those in the upper third on ability. Thus, ability but not sex served 
as a moderator variable in this instance. On the basis of results obtained 
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in the above and other studies , it would appear that Ghiselli ^d Seders 
observation (196T) that moderator variables are situation-specific is 
very appropriate. 



Robert and Dunnette (196?) continued the reformulation of the 
prediction problem begun by Dunnette (1963). Using what was ca e e 
quadrant-analysis technique, they divided the bivariate, predictor-criterion 
frequency distribution at the predictor median. The predictor , in is 
case, consisted of a composite score on a comprehensive battery of 
graphical, personality, ability, and managerial judgment measures, 
same biographical data and personality variables were then used to dif- 
ferentiate subjects above and below the median on a criterion 
of a comprehensive measure of managerial effectiveness . epara e 
scales were developed for the "high predictor group (above pre ic or 
median) and the "low" predictor group. These moderator scales were then 
used to identify and eliminate overpredicted indiv^uais (lower rig 
quadrant) and underpredicted individuals (upper left quadrant) from^ 
cross-validation sample. Of 220 cases in the sample, 2 % were eliminated 
as unpredictable. The point biserial correlation ^etwe^ the predic or 
and criterion in the cross-validation sample was .73. comparable 

figure for the total cross-validation sample was . 65 . Thus a substanti 
increase in validity was accomplished at the expense of some reduction 
in sample size due to the elimination of the least predictable cases. 

One cannot help wondering, however, whether comparable results could 
have been achieved by simply eliminating the middle 25 % of the cases on 
the predictor variable. Robert and Dunnette think not but no data were 

provided. 

There is another way in which to look at the Robert and Dunnette 
data. Suppose the predictor variable is viewed as the moderator variable, 
in this case dichotomized at the median. Suppose, then, criterion scores 
above the criterion median are considered to be "successes and ose 
below as "failures." One then has success-failure groups with high 
predictor (moderator variable in this case) scores and similar groups 
with "low" moderator variable scores. In the process of developing what 
were called moderator tests , Robert and Dunnette were really forming 
separate scoring keys for the high-low groups . The procedure veiy 
similar to that reported by Prediger (1966). Rowever, m the study by 
Prediger, application of the results of the px«-.cedures was 1 ere 
as will be discussed below. 

Earlier work ty Prediger (1965) on the relationship of biographical 
data (biodata) to persistence in college led him to hypothesise that the 
relationship might actually be moderated by student characteristics such 
as academic aptitude and prior achievement. Using two-ye^ persi t 
data for lU69 University of Missouri entering freshmen males and » 
biographical inventory constructed specifically for ® 

( 1966 ) demonstrated that academic ability is, in fact, an effectiv 
moLrator variable in the prediction of persistence from biodata. Sepa- 
rate scoring keys were developed for the biographies •>,„+< on 

of three ability levels . For two of the ability groups the contribution 
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of biodata to the differentiation of persisters and ^ 

sssriSSw' »» 

ment of moderated scoring keys was demonstrated. This approach ^ 
allows for the identification of subgroups that have greater or le 
S^eL of predictability, which is as far as the previous approaches 
developed "by Ghiselli and others have gone. 

Sophisticated techniques for the identification of 
been reviewed hy Dunnette (1963) and most recently demonstrated oy 
SchLnfeldt (1966). Rock, Barone, and Linn (1967) describe a c^prter 
pro^^ thL groups individuals on up to five “operator v^iaoles in 
such a fashion that the relationship between ten 

criteria is optimized for the total sample. However, for the purp 
of investigating ways of refining moderated scoring key techniques, the 
iLfl^beS^e pr^edures for identifying ability level subgroups dis- 
c^s^d in the section on design would appear to be adequate. 

Before experimentation with moderated scoring keys can take Pla=e, 
a relatSLS must be present between items in the inventory for which 
Lyf arnfbe developed and the event or characteristic to be Predicted. 

In this study, biodata was ^ed ^ a f 

of college entrance is one that has been frequently shown to be related 
to various Spe“rof biodata. Such relationships have already been 
reported for a few Project TALENT biodata items (Flanagan and Cooley, 

1966). 



Objectives 

The primary objective of this project f ® ^ 

■efinement of the techniques for developing moderated keys first reported 

,y Prediger (1966). Investigation centered in two areas: 

(a) Development of procedures for deteimining the optimum number 
of sub^oups (and hence, moderated scoring keys) required for 
maximizing the predictive effectiveness of an inventory. 

(b) Development of a single scale for °^hf 

from a set of moderated scoring keys , thus 

practical application of moderated keys to prediction problems . 

fl+T nns Conv6rsion to standard scoreb rep 
tr^rercwve implications of comparable standard scores on different 
moderated scoring key scales are likely to differ greatly. 
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The secondary objective of the pro.iect involved investigation of 
the generalizability of moderated scoring key techniques. In order to 
achieve this objective and at the same time evaluate the effectiveness 
of procedures developed through pursuit of the primary objective, the 
following general and subsidiary hypotheses concerning the application 
of moderated scoring keys to the prediction of college attendance were 
made: 



(a) General hypothesis — ^The contribution of biodata to the pre- 
diction of college attendance is greater upon cross-validation, for 
scoring keys developed within ability level subgroups (£.£.. , moderated 
keys ) them for a key developed on the total group j the general 

^T. 



(b) Subsidiary hypotheses — Moderated scoring key predictions of 
college attendance-nonattendance for high school students are more 
accurate than predictions based on: 

(51) the normal empirical keying procedures used in forming the 
general key. 

(52) the above, plus academic aptitude data in an optimally weighted 
combination. 

(53) academic aptitude data alone. 

In order for the general hypothesis to be supported, it is necessary 
to confirm only subsidiairy hypothesis SI. However, S2 and S3 are directly 
relevant to the practical application of moderated keys based on biodata 
sines these keys must produce more accurate predictions than could be 
obtained through other readily available means . In otner words , there 
is no substitute for incremental validity . 



Procedures 



Population and Sample 

All data used in this study were obtained from the Project TALENT 
Data Bank. The subjects constitute a subgroup of the stratified random 
sample of the U. S. high school seniors who participated in the data 
collection phase of Project TALENT in the spring of I 96 O. Stratification 
was based on factors such as type of school and geographical area so 
that the sample would be representative of all high school seniors in 
the U. S. The nature of the stratification variables and the representa- 
tiveness of the obtained sample are discussed in a publication entitled 
The Project TALENT Data Bank (1965). 

Of the total senior sample, only males who responded to the Project 
TALENT follow-up questionnaire mailed in the summer of I 96 I and for whom 
data were available on the Project TALENT Data Bank tape files in July, 
1966 , were included in this study. (Females were not included because 
the relationship between their biodata responses and the criterion is 
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likely to differ fran that of males, thus necessitating separate analyses.) 
Approximately 30^ of the seniors in the i960 sample failed to respond to 
the follow-up questionnaire (Flanagan and Cooley, 1966). Since important 
differences between respondents and nonrespondents were found to exist, 
the subjects included in this study cannot be taken as representative of 
the U. S. popiaation of high school seniors. Screening for missing data, 
which is described in the section on design and analyses, further reducea 
the representativeness of the sample. 

It should be emphasized, however, that for the purposes of this study, 
representativeness of the population of U. S. high school seniors is not 
essential since generalization of this population is not intended. Instead, 
the relative effectiveness of the various keying techniques within the avail 
able sample is compared. The sample used in this study should represent 
reasonably well a hypothetical population of high school senior males who 
choose to respond to the type of follow-up procedure used by Project TALENT. 
While it may be useful for readers to think in these terms, it is not 
essential to interpretation of results. Of the 21,53^ cases suppli^ by 
the Project TALENT Data Bank, 20,367 were available for the study after 

screening was completed. 



Variables 

A wide variety of data was collected on Project TALENT students during 
a two-day period in the spring of I96O. Of the approximately 100 septate 
test and inventory scores which were available, the following were selected 
for possible use in a composite moderator variable: Vocabulary , 

EnglLh Grammar Total (230); Reading Comprehension ( 250 ) ; Abs-^act Reasoning 
(290); Arithmetic Computation (UlO); and Mathematics (320). number in 

parentheses after each variable precisely identifies the variable according 
to the Project TALENT code. The variable titles identify them as 
measures of academic aptitude and achievement. Each is further described 
in a manual entitled The Project T ALENT Data Bank (1965). 

Two of the above six variables, Reading Comprehension and Mathematics, 
were selected for use in the moderator variable. Selection w^ ^e 
three factors: (a) point biserial correlation of the variables with college 

entry, otherwise undefined, as revealed in preliminapr data developed by 
Project TALENT (Flanagan, et ^. , 196U); (b) app^ent adequacy o 
range, floor, and ceiling of the measures as indicated by Project ^ avail- 
normative data (Flanagan, et a.1 . , 196^); and (c) e num er o , . 
able for each possible combination of two varia^ es ^ ® data 

preliminary editing of the magnetic tapes containing e P 
The correlation between Reading Comprehension scores and entry into college 
between high school graduation and the one-year follow up w^ repor e 
.h2 while that for Mathematics was estimated to be M on the basis of 
this variable's similarity to others for which data was reported, ^ese 

were the highest correlations reported for the six varia es . ^ ^ ctndipd 
the best predictor of college entry among the 56 cognitive variables studied 
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was the Mathematics Information Scale (R-106) with a correlation of .52. 
This scale, although less adequate in terms of range and ceiling, was 
highly related to Mathematics (320). Finally, the tape edit revealed 
that loss of cases due to missing data for these two variables would be 
slight. The edit sample is identical to analysis group 1 which is 
described in the section on design. 



In order to form the moderator variable, Reading Ccmprehension and 
Mathematics scores were combined by use of linear standard scores. Means 
and standard deviation {33.30h and 9-59T for Reading Comprehension and 
2I+.II8 and Q.hhh for Mathematics) were obtained on the editing run. These 
were used to obtain standard scores with a mean of 500 and a standard 
deviation of 100 for each variable. The composite moderator variable 
was the sum of the: two standard scores . 

As can be seen, an effort was made lu construct an ability moderator 
that would include commonly used academic aptitude variables and, at the 
same time, would have a substantial relationship with the criterion to 
be predicted. A different approach might have involved the use of a 
moderator variable more or less unrelated to the criterion. The relative 
utility of the two approaches is unknown. However, with an item pool 
consisting of biodata, the criterion-relevant ability moderator appeared 
to be an ideal choice. 



The biodata used in the study was obtained frm the Student Informa- 
tion Blank (SIB), a questionnaire with items covering topics such as 
hobbies ; club memberships ; social activities ; health ; work experiences , 
study habits; school experiences; characteristics of family, home and 
community; parents* age, source of income, activities, and education; plans 
for college, military service, marriage, and career; e^. Of the 39^ 
items, ihQ were eliminated because they did not appear to fit the common 
definition of biographical data. These items chiefly covered study habits 
and plans of various kinds . 



Since the exact nature of the 2k6 item biodata pool is not relevant 
to this study, the full, copyrighted instrument is not reproduced h®ye. 

A copy can he found on pages 5-6 through 5-35 of the publication entitled 
The America n High School Student (Flanagan, et al . , 1961*). Item numbers 
for the items that were used and the identification number assigned to 
them for purposes of this study can be found in Appendix A. Since 14H of 
the original items were not used and since some rearrangement of items 
was necessary for processing, the project item identification number 
differs in many cases from the SIB number. Including omits , a total of 
about 1650 response options were available for item analyses. Although 
item analysis data were obtained for omits , omits were not included on any 

scoring keys. 



Of the 2h6 SIB items that were used in this study, 226 contained from 
two to six response options while the rest contained from 10 to IT options. 
To facilitate analyses, these latter items were ^structured so as to 
require no more than nine response categories. This involved combining 
response options that had low frequencies as reported by Project TALENT 
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and/or that seemed to fit together on a logical "basis. Because of the 
unusually large number and diversity of response options for SIB items 
206 and 208 , each of these two items was split into two parts with each 
nart treated as a separate item. The analyses for each response involved 
all cases in the item analysis sample, not just those of students marking 
responses in a given part . A list of all combinations and the response 
option identification numbers used in the study is presented in Appendix 
B along with some minor alterations of response options for three other 
items. As can "be seen in Appendix B, for purposes of this study, the 
response options were indicated hy numbers rather than letters (as on 
the sib). Thus, response option A was assigned to number I; B was assigned 
2, etc , in all cases except where recoding was required. 

The criterion variable, college attendance, was obtained from the 
responses of students to item two on the 1961 follow-up questionnaire. 

A copy of this questionnaire is presented in Appendix H of the America n 
High School Student (Flanagan, et al . , 19^h) . Item two is reproduced in 
Appendix D of this report. Those students indicating that they had 
entered college as full-time students are included in the college atten- 
dance group. All others are in the nonattendance group. Thus, the 
students’ responses to item two served to operationally define the criterion, 
college attendance. It is recognized that some students indicating college 
attendance may have dropped out after a short time and that others not 
indicating attendance may enroll in subsequent years. In addition, part- 
time students may become full-time students and vice versa. However, the 
above definition of college attendance proved adequate for purposes of this 
study . 

Design and Analyses 

As was already noted, the editing run on data supplied by Project 
TALENT revealed missing data in some of the student records . Obviously , 
if a student’s response to item two of the follow-up questionnaire was not 
available, the rest of his data was lost to the study. In addition, scores 
for the two tests used in the moderator variable had to be present and 
within the possible raw score range. Finally, SIB responses had to be 
present for analyses to be worthwhile. Hence, in addition to the other 
checks, each record was checked, before processing, for the presence of 
responses for the first ten SIB items. If no responses were present, the 
record was ignored and hence was lost to the study. Approximately ^00 
records were lost for this reason. An additional UOO were lost because 
of lack of the required test scores. Finally, foxlow— up data for item 
two was not available for about 360 cases. The records of 20,36? students 
remained after these checks were made. For purposes of analyses, these 
records were randomly subdivided into four independent groups . 

Before any analyses could take place, a procedure had to be developed 
for detemnining the optimum number of subgroups (and hence, moderated 
scoring keys) required for maximizing the predictive effectiveness of the 
item pool. (See primary objective number one.) The only feasible approach 
involved the subdivision of the item analyses sample into ability groups 
according to score on the moderator variable. Using Project TALENT figures 
on attend, nonattend percentages at various ability levels, the effect on 
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criterion group size of various numbers of ability level groups was 
determined in light of the need to allocate the total sample into four 
subgroups for analyses. To be avoided were extreme attend, nonattend 
splits and low sample sizes within an ability level group. This, of 
course, would most likely occur at the extremes of ability. 

Twelve ability level subgroups as nearly equal in size as possible 
were chosen as being appropriate to the purpose of the study and the data 
available. By combining response frequencies for adjacent groups, item 
analysis data based on six and three ability level groups could be 
obtained without need for a separate item analysis. In addition, data 
for the total sample could easily be obtained by combining data from the 
three ability level groups, each of which represented four of the original 
twelve groups. These were important practical considerations in light of 
the large sample size and number of response options to be analyzed. 

By use of the 12 ability level subgroups, scoring keys could be 
formed for the total group and for 3, 6, and 12 subgroups. Thus investi- 
gation of the optimum number of moderated scoring keys required for 
maximizing the predictive effectiveness of the item pool was possible. 
Procediares for dividing the sample into twelve subgroups are described in 
the section on Group 1 analyses. The natiire of the keys is also described 
in that section. Procedures for deciding on the optimum number of moderated 
keys are described in the section on Group 3 analyses. 

Group 1 Analyses . The records of 10 ,183 students were randomly 
assigned to this group. A computer program was written to calculate the 
moderator variable score for each student in the group and to determine 
the score limits that divided the group into twelve subgroups as nearly 
equal in size as possible. Tie scores and the fact that 10,183 is not 
evenly divisible by 12 prevented the groups from being exactly equal. 

Score limits and frequencies for the twelve groups are shown in Table 1 
along with the attend, nonattend split in each of the groups. These 
score limits were used in all subsequent analyses. 

Since moderated scoring keys were to be formed at each of the twelve 
ability levels and combinations thereof, 21 moderated keys had to be 
developed. These plus the general key made a total of 22 keys . Four 
keys were appropriate to each of the 12 ability levels: the general key, 

the key specific to the ability level , the first order combined key 
formed with data from two adjacent levels and the second order combined 
key formed with data from adjacent first order combined keys. The keys 
appropriate to each of the 12 ability levels are shown in Table 2. As 
can be seen, key 22, the general key, was formed and scored on all students. 
Normal empirical keying techniques would have involved only this key . 

Group 1 was used to obtain the item analysis data from wh. ch all 
scoring keys were developed. For each response to an SIB item, a two-by— 
two contingency table was formed using marked-not marked and attend- 
nonattend dichotomies. Phi coefficients calculated from this data formed 
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Table 1 



Moderator 


Variable 


Score 


Limits for the 


12 Ability Level Groups 


Ability 

level 


Score limits 


Frequency 

Attend Honattend 


Total 


1 


1^01 - 


718 


106 




850 


2 


719 - 


810 


162 


681 


8I43 


3 


8 ll - 


872 


228 


628 


856 


k 


873 - 


928 


275 


566 


8ia 


5 


929 - 


976 


3hl 


511 


852 


6 


977 - 


1021 


hO'J 


1435 


8142 


7 


1022 - 


106i^ 


h59 


hO'5 


862 


8 


1065 - 


IlOh 


518 


326 


Qhh 


9 


1105 - 


lli ^7 


596 


256 


852 


10 


lli ^8 - 


1193 


6h2 


208 


850 


11 


119^ - 


1238 


671 


151 


822 


12 


1239 - 


1600 


777 


92 


869 


Total 






5182 


5001 


10183 



Kobe . —Limits and frec[uencies based on data from analysis group 1 
Maximum and minimum possible moderator variable scores were l600 and 
respectively. No scores reached these limits. 
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Table 2 



Keys Appropriate to Each of the Twelve Ability Levels 



Ability 

level 


Moderated scoring keys 


General 

key 


Ability 
level keys 


1st order 
combination 


2nd order 
combination 


1 


1 


13 


19 


22 


2 


2 


13 


19 


22 


3 


3 


lit 


19 


22 


k 


k 


ih 


19 


22 


5 


- 5 


15 


20 


22 


6 


6 


15 


20 


22 


7 


7 


l6 


20 


22 ■ 


8 


8 


l6 


20 


22 


9 


9 


17 


21 


22 


10 


10 


17 


21 


22 


11 


11 


l8 


21 


22 


12 


12 


18 


21 


22 



lyjote. Ability level 1 represents the lowest ability group. 
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the basic item analysis statistics although chi squares vrere also obtained 
along with the proportion of the attend and nonattend groups that marked 
each response option. As previously mentioned, omits were treated as 
reponse options in the item analyses although this option was not included 
on any scoring key. 

Before keys could actually be formed, a procedure for determining 
key length had to be developed. In the past, a common item analysis 
practice has been to key only those responses which show a relationship 
with the criterion that is statistically significant at some predetermined 
level, usually the five per cent level or the one per cent level. However, 
the results of studies by Appel and Kipnis (195^), Kuder (1951), and Clark 
(1961), suggest that the optimum significance level cut-off point is a 
ccxnplex function of the criterion, the size and nature of the sample, and 
the test or inventory being used. Most recently, Abrahams (I96T) showed 
in a convincing fashion that optimum key length is specific to the item 
pool being keyed. Hence, it was necessary to develop keys of varying 
length for each ability level subgroup and the total group so that it 
would be possible to determine empirically the optimvim key length upon 
cross - vali dat ion. 

Of several possible approaches , the one chosen involved the use of 
five phi values as cutoffs for keys of five different lengths. These phi 
values were .03, .05, .07, .10, and .15. Thus the longest subkey at a 
given ability level would be made up of item responses with phi coefficients 
of .03 or higher. The shortest subkey would use .15 as the cut off. This 
was done for each of the 22 keys. A total of 110 subkeys were formed, 5 
for each of the 22 keys. 

Obviously, with the volume of item analyses data involved, some 
mechanical means of forming keys was required. No attempt to identify 
logical or illogical response validity data was made. Instead, a com- 
puter program was written to form subkeys. It was applied in the same 
manner to itert analysis data for each of 22 keys . 

Of the 110 subkeys formed in this way , those appropriate to keys 
that were actually scored after preliminary analyses are presented in 
Appendix C. For each key and subkey length, the item and response mmiber 
is identified along with the direction in which the response is to be 
scored, i_.e . , +1 or -1. Thus Appendix C really represents an index to 
item responses with phi coefficients falling in five different intervals 
of absolute value: .03-. 0^9? .05-.069j .0T-.099) .10-. 1^9? and .15 or 

greater. This manner of presenting item response validity data rather 
than a complete listing of item analysis data was chosen because the 
latter would require more than 6OO pages. 



Anyone interested in which items had a given level of relationship 
with the criterion should refer to Appendix C. It is this data, along 
with the results obtained in determining optimum key length and number 
of moderated scoring keys , which could be used in constructing an instru- 
ment to identify students who are likely or who are not likely to attend 
college . 
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Group 2 Analyses . The records of 338? students were randomly assigned 
to this group. Each student was placed in one of the 12 ability level 
groups on the basis of his moderator variable score. The SIB responses 
of the students were then scored on four keys (see Table 2), each with a 
total of 5 subkeys of varying length. Thus twenty scores were obtained 
for each student. All scoring was done by computer and at the s^ie tinie, 
data was accumulated that allowed the calculation of point biserial key- 
criterion correlations for each of the 110 subkeys (22 keys, 5 subkeys per 
key). The trend in these point biserials across the five subkeys scored 
for a given key was used to estimate optimum key length for that key. 

Only this key length was used in subsequent scoring. 

Group 3 Analyses . Analyses on this group had three primary purposes: 
(a) to determine the optimum number of moderated scoring keys appropriate 
to the data; (b) to develop a common score scale on which key results 
could be reported; and (c) to develop equations for use in predicting 
college attendance versus nonattendsmce for students in Group the 
cross-validation group. As explained below, it was possible to accom- 
plish steps 2 and 3 at the same time. The records of 3393 students were 
randomly assigned to Group 3 analyses. 

jn order to detennine the optimum number of moderated scoring keys , 
some way to compare the effectiveness of different numbers of keys had 
to be devised. Comparison of the point biserial correlations obtained 
for the sets of 12, 6, and 3 keys was not satisfactory because the atten- 
dance, nonattendance split differed across the sets. For example, the 
split in group 12, the highest ability group, would not be the same as 
the split in groups 9 through 12 combined. As discussed by Guilford (195 )» 
the point biserial correlation coefficient is sensitive to the criterion 
split. As the split becomes more extreme the values reached by the co- 
efficient tend to be depressed, other things being equal. Hence, the 
point biserial coefficients could not be used in any simple fashion for 
interkey comparisons. 

It was appropriate, however, to score each higher order key combina- 
tion on the next lower order combination. Thus, the general key could e 
scored on the ability groups appropriate to keys 19, 20, and 21. If the 
development of moderated scoring keys is of no particular value, then key 
22 should do as well on the groups appropriate to keys 19, 20, and 21 as 
the keys do themselves. In fact, it should do better because 
on an item analysis sample three times as large as the second or er ey 
combinations. At the same time, the effect of criterion group split is 
controlled because all point biserial comparisons are made on t e same 

group . 

Using this principle, the following rules were devised for determining 
optimimi key length. 

1. The general key is scored on the groups appropriate to the 
second order combination keys, keys 19, 20, and 21, and will replace 

any of these three keys iinless the difference in point biserial corre a- 
tion coefficients is at least .01 units in favor of the second order keys. 
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2. The second order combination keys resulting from step one are 
scored on the groups appropriate to the first order combination keys, 
i.e., keys 13 through l8. Each second order key is scored only on the 
two first order combination groups appropriate to the key. Thus key 19^ 
would be scored in key groups 13 and lU separately. If the point biserials 
for a second order key are higher than those of both first order keys, the 
second order key will be used instead of the first order keys. If the 
point biserials are lower than those of both first order keys, the first 
order keys will be retained. If the results are mixed, i.e., second order 
biserial hi^er than one of the first order key biserials but lower than 
the other, a judgment will be made as to which difference is greater. For 
the sake of economy in scoring time, this decision will generally be made 
in favor of the second order keys. If the decision is in favor of the 
second order key, it will replace both first order keys. If not, both 
will be retained. 



3. The first order combination keys resulting from step two are 
scored on the ability groups appropriate to those keys. Thus key 13 (or 
its replacement) would be scored separately on ability groups 1 and 2. 
Decision rules paralleling those in step two are then applied. 



Application of these rules would appear to provide a flexible as 
well as workable procedure for determining the optimum number and combina- 
tion of moderated scoring keys. It would be possible for the general key 
to replace all of the moderated keys or for some combination of keys in- 
volving the general key , second order keys , first order keys , ^d ability 
group keys to be selected. For example, ability groups 1 U might be 
scored on the general key, groups 5—8 on key 20, groups 9--^0 on key IT, 
and groups 11 and 12 on their own moderated scoring keys. This would be 
an example of a case where moderated scoring keys are unimportant for 
low ability groups but become more and more important as ability increases 



Once the optimum set of keys was deteimined, a common score scale had 
to be developed along with equations for predicting attendance '^’ersus 
nonattendance. Both of these problems were solved through ^e 
classification procedure described by Cooley and Lohnes (1962) . This pro- 
cedure allows one to determine the similarity of an individua o one or 
more groups on the basis of one or more scores for e in i.vi ua . n 
addition to differences in group means, differences in group size and 
dispersion are taken into account. In this case there were two groups, 
college attenders and nonattenders , for which a student s similar iy w^^ 
to be estimated. These estimates of similarity were in e orm p 
abilities ranging between .00 and 1.00. 



Prediction equations were developed from group 3 data for each of the 
keys in the best combination of moderated keys, for the gener ey, or 
the ability moderator, and for the general key and ability moderator in 
optimally weighted canbination. Ibe estimates of attendance g^up simi- 
larity obtained from the equations for moderated key scores formed a 
common score scale to which the moderated keys could be anc ore 



Group k Analy sis . Analyses conducted on this group were for the 
purpose of obtaining information bearing directly upon the gener an 
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subsidiary hypotheses . The records of SboU students were randomly 
sssigned to those an&Xyses# 

Scores for each student were obtained on the moderated scoring key 
appropriate to his ability level, the general key, ^d the moderator 
vLiable. Prediction equations developed in Group 3 were 
these scores so that for each student estimates of group 
on the three scores mentioned above plus the last two in optimally 

wei^ted combination were obtained. Thus, ^in the 

frrow similarity estimates for each student. These estimates, 
form of probabilities, were based on (a) student’s score on he 

moderated scoring key appropriate to his ability f ^ 

on the general key; (c) his score on the general key optim^y ^orbinea 
with his academic aptitude score; and (d) his academic aptitude score 
alone. Whenever the probability exceeded .50, attend was pre 
Otherwise, "nonattend" was predicted. 

Records of hits and misses according to what the students had 
actually done were accumulated for each of 

the scoring was done. More important, a record of h jLLi-- 
data fHL moderated keys and each of the other es 

maintained. In this way it was possible to deteiuane the of c^es 

for which the general key and moderated key , for exampl , p 

both "hit" or^oth "missed." More important, i* ^ 
to determine the number of cases for which one pre ic or 
other missed. If the moderated keys correctly predicted ma^ of the 
:^:s Tich ihe general key missed but the reverse fid not happen the^ 
the moderated key would clearly be superior. 

by McNemar (1962) for data of this kind was used to ^ f ^J^^ig^ical 

the information of those interested in the traditional tes 

significance. 

The same procedure was followed for each of the subsidi^ hypotheses, 
In addition nhi coefficients expressing the relationship between pr 
dieted criterion group membership and actual criterion group mem ers ip 
were obtained for each of the four predictors . 



Results 



Group 1 Analyses 

AS explained in the section on design, data for this group_w^e used 
in the formation of ability group limits (see oporlne keys are pre- 

analysis, and in the fL’five subkeys for each 

oftL 22 keyris pr;sented in Table 3. ?te®IppearScf 

length for all keys thus providing ample o^ortmity 

of trends in the ''^l^tionship between key^e^ n y^ ^ analyses, 

elusions relevant to project objectives can oe maue 
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Table 3 



Key Lengths for Each of the 22 Keys 



Kev 


.15 


Ko. of resnonses scored per phi cutoff 


.10 


♦OT 


.05 


.03 


1 


h 


52 


153 


3h'{ 


6T1 


2 


11 


56 


199 


391 


69T 


3 


22 


ll6 


283 


502 


T85 


k 


Ik 


92 


251 


h33 


T8o 


5 


20 


89 


23T 




T62 


6 


20 


105 


281 


U91 


815 


T 


20 


115 


293 


5IT 


831+ 


8 


30 


126 


29T 


506 


808 


9 


15 


T6 


19T 


399 


TOO 


10 


21 


100 


265 


U58 


T65 


11 


1 


6T 


22U 


U35 


tuu 


12 


9 


36 


ITO 


338 


62T 


13 


h 


35 


110 


260 


5T2 


lU 


9 


81 


218 


hob 


6t8 


15 


16 


82 


198 


395 


T33 


16 


19 


99 


259 


UU5 


T30 


IT 


13 


6T 


186 


3U8 


650 


18 


2 


31 


136 


289 


601 


19 


6 




156 


303 


580 


20 


IT 


Ih 


210 


385 


676 


21 


8 


52 


156 


30T 


590 


22 




159 


326 


50T 


80 U 
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Group 2 Ansilyses 

Analyses on Group 2 involved the determination of optimum key length 
for each of the 22 keys . The results of these analyses are presented in 
Table As can be seen, optimum key length and peak pattern varies from 
key to key. Although much of this variation may be due to chance effects 
in item and group sampling, trends for the larger groups should be 
relatively stable. For the type of data used in this study, most key 
validities appear to reach a plateau rather early. In this connection, 
Abrahams (196?) has noted that for an item pool with a given range of 
response validity, there is very little increase in key validity after 
100 to 150 item responses have been combined. Key lengths used in all 
subsequent scoring are underlined in Table 

As can be seen in Table results for 110 subkeys were obtained. 

The SIB responses of each student were scored on ^ keys (20 subkeys). 

These four keys included the general key scored on all students and thr* c 
keys selected according to the student's ability moderator score. For h 
of the 20 subkeys, more than 700 item responses were often scored. And 
yet, with the FORTRAIv program that was written, scoring proceeded at the 
rate of less than one second per student on an IBM 360 Model hh. Data on 
key statistics were accumulated as the scoring was done. Hence, the type 
of analyses performed would appear to be well within the limits of practi- 
cality . 

Ho conclusions relevant to project objectives can be made from Group 
2 analyses. 

Group 3 Analyses 

As described in the section on general design and analyses, rules 
were fonnulated for determining the optimum number and combination of 
moderated scoring keys required for maximizing the predictive effective- 
ness of the item pool. These rules were applied to the point biserial 
correlations presented in Table 5* As can be seen, the general key was 
retained for scoring at eight of the twelve ability levels. Keys 17 and 
l8, both first order combination keys, were retained at ability levels 
9-10 and 11-12, respectively. Thus, it would appear that ability serves 
as a moderator variable only for the more able students . Even for this 
group, the effect is not great. 

When applied to real data, the procedures and rules developed for 
determining the optimum number of moderated scoring keys seem to work 
well. It would appear that both the procedures and the rules can be 
readily applied to other data if sample sizes are substantial. Hence, 
on this basis, it is concluded that primary objective number one has been 
achieved. 

Results at this stage of the analyses make it doubtful that subsidiary 
hypothesis SI will be confirmed. However, Group h analyses are required 
before a conclusion can be reached. 

Classification procedures used for developing a conmion score scale 
in the form of probabilities of college attendance group membership were 
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^ ^ ^ I. 



Relationship Between r.ey Ler.ftr. and 
for Bach of the 22 Keys 



Key Validity 





Saniple size 




^nt bis 


per phi 


cutoff 




Key 


Attend 


Konattend 


.13 


.10 


.07 


.05 


.03 


1 


32 


25c 


.197 


.182 


.157 


.lop 


.180 


2 




220 


.235 


.29I 


.258 


.272 


.270 


3 


59 


165 


• ^ 


.328 


.325 


. 3"22 


.329 


1* 


96 


187 


.ii05 


.222 


.223 


.226 


.232 


5 


117 


168 


.ifbO 


.255 


.257 


.263 


Mb 


6 


155 


il?l4 


.333 


. 375 


.370 


.367 


.392 


T 

8 


iJiT 

15^^ 


136 

108 


.330 

Mn 


.213 


. 207 
.257 


.399 

.2;5 


.398 

.2?6 


9 


206 


60 


.383 


•^f'n 

• V. ^ 


.321 


.322 


. 320 


10 


229 


78 


»3oT 


. 332 


.387 


.382 


.377 


11 


230 


55 


.135 


.321 


. 330 


.315 


.322 


12 


261 


37 


.295 


.3ir 


.317 


• 38:5. 


.292 


13 


77 


I475 


.2t:''- 


.251 


252 

*5 , '♦■ 


.221 


.272 


lU 

15 


155 

272 


372 

512 


.3ifl 

'/'7 

• 1 


. 3cd 

. 208 


.202 

.218 


.21? 

.i^8 


.205 
. 226 


l6 


301 


2UI4 


. 327 


.227 


.2u0 


.23*6' 


.2 


IT 


if 35 


158 


.292 


. 359 


.357 


.378 


.369 


l6 

19 


U91 

232 


92 

850 


.152 

. 3 


. 32 0 

. 350 


.321 

.365 


.329 

.36i 


4ij 

•39_8 


20 


573 


556 


.392 


.223 


.256 


.229 


. 252 


21 


926 


250 


■;liv 


.377 


. 390' 


^ 

•3?ji 


. poy 


22 


1731 


1656 


.520 


•ITk 


.506 


.565 


.556 




I.-ote. — The 


key len/'th that 


war. used 


in subsepiuent 


scorinp. 


is 



underlined. 
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Table 5 



Determination of Optimiun limber of Scoring Keys for Group 3 



Key 

group 




Point biserlal correlations 




Sample 


size 


Appro- 
priate key 


Higher order key 
22 21 20 19 18 


17 


Attend 


Don- 

attend 






Step 1^ 








21 


.372 


.357 




931 


242 


20 








538 


593 


19 


,h 06 


.hob 




242 


647 






Step 2^ 








18 


.278 


.2hg 




461 


76 


17 


.UlO 


.hl3 




450 


166 


lb 


.h23' 






303 


2bl 


15 


.i^57 


.479 




235 


332 


lU 


.w 


.460 




156 


370 


13 


.2hh 


•ki5 




66 


469 






step 3^ 








12 


.2h0 


.248 




259 


27 


11 


.258 






222 


49 


10 


.300 




.348 


261 


73 


9 


.hG9 




•k57 


169 


93 


3 


.359 


.421 




169 


125 


7 


• U55 


. 50 8 




134 


136 


6 


.U54 


.473 




131 


158 


5 


• lie 


-. 470' 




104 


174 


k 


.503 


.470" 




77 


193 


3 


.^a7 


.456 




79 


165 


2 


.137 


.218 




55 


233 


1 


.251 


•83b 




31 


236 


iiote. — Keys ch 


osen at eacn step are underlined. 









^Refers to steps 
design. 



for detei^-ininr optin.un key lenrth. 



See section on 
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described in the section on general design and analyses. These procedures 
require data on criterion group sizes, means, veiriances, and, where two 
predictors are used in combination, covariances. This data is presented 
for analysis Group 3 in Table 6. Standard deviations are shown instead 
of variances. In addition, ^ test data is presented for each of the 
point biserials. As can be seen from these ;t values, it is highly unlikely 
that any of the correlations could have been obtained by chance. 

The common score scale used in this project would appear to satis- 
factorily meet the requirements of primary objective two. In principle, 
it involves the use of criterion predictions as a common basis for anchoring 
the various moderated scoring key score distributions. This procedure 
should work equally well whether criterion group membership or criterion 
scores are being predicted. In the latter case, regression rather than 
classification procedures could be used to form the ccmimon score scale. 

Under the suggested system, the score reported for an individual on 
a moderated key is essentially a prediction directly relevant to the 
purpose for which the key was constructed and is to be used. Whenever 
data are available on the actual predictive validity of an instrument 
for the purposes and group on which it is to be used, it should be 
possible to develop a common scale for reporting moderated scoring key 
results. Hence it is concluded that primaiy objective two has been 
achieved. 

Group U Analyses 

The main purpose of the Group ^ analyses was to determine the 
accuracy of attendance, nonattendance predictions so that the relative 
effectiveness of the four predictors developed in this study would be 
revealed. The results of these analyses are directly relevant to achieve- 
ment of the secondary objective of the project, and hence to testing the 
general and subsidiary hypotheses. 

Data summarizing the relationship between predicted and actual 
status on the criterion are summarized in Table 7 for each of the four 
predictors. It is immediately apparent from the hit rates djid phi values 
that, in this particular situation, use of moderated keys hold no parti- 
cular advantage over use of the general key. The phi coefficient (.5^) 
and hit rate (77%) for the moderated key set were both only slightly 
higher than the phi coefficient (.52) and hit rate (7o^) obtained with 
the general key. Thus, academic aptitude does not appear to be an 
effective moderator variable for the predictor and criterion under study. 

The same data are presented in a different form in Table 8 where 
joint hit, miss rates are shown. The most interesting aspects of this 
data are the instances in which one predictor made a correct prediction 
(a hit) while the other yielded a miss. For example, the moderated keys 
correctly predicted 135 of the general key misses while the general key 
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Table 6 



Descriptive Data for the Optimum Set of 
Moderated Scoring Keys for Group 3 



Ability 

group 


Key 

used 


Attend 
K X 


SD 


Nonattend 
N X 


SD 


^pt bis 


- 

t 


1- 8 


22 


T80 


91.1 


10.6 


lUUO 


78.2 


10.6 


.50 


27.3 


9-10 


IT 


U50 


188.9 


15.6 


166 


173.1 


15.^ 


M 


11.1 


11-12 


18 


l;8l 


186.7 


13.0 


76 


175.7 


13 .h 


.28 


6.8 


1-12 


K22 


1711 


96.1 


11.3 


1682 


80.0 


11.5 


.58 


1*1.1 


1-12 


MV 


1711 1096.6 


1U5.3 


1682 


917.0 


171.6 


.1*8 


32.0 



Note.— For purposes of information, data is also presented for the 
general key "(K22) and for the moderator variable (MV). The covariance for 
K22 and MV in the attend group was bll.6. For the nonattend group the co 

variance was 921.^ • 



23 



o 

ERIC 



Table 7 



Comparison of Accuracy of U Predictors in the Prediction of 
College Attendance for Group U 



Actual status 


Predicted status 
Kon- 

attend Attend 


Hit 

rate 


Phi 






Moderated Keys 








Attend 


386 (n>) 


13^+3 ( 39 fO 








Non-attend 


1273 ( 31 %) 


h02 (12%) 














11% 




981.1 




General Key 








Attend 


U17 (12f.) 


1312 (38%) 








Mon-attend 


1278 ( 3 Q%) 


397 ( 12^0 














16% 


.52 


926.6 




Moderator Variable 








Attend 


372 (11%) 


135 T (*tOS) 








rion-attend 


1056 ( 31 >) 


619 (18;;) 














11% 


.h2 


602.6 


Best Combination of General Key and 


Moderator 


Variable 


Attend 


379 (11/0 


1350 (I4O/O 








Hon-attend 


126U ( 37/0 


hll (12^0 














11% 


. 5 ^ 


976.8 



Wot e.— Per cent of total K given in parentheses is rounded. Hit 
rate represents a separate rounding of totals. 



2h 



O 

ERIC 



Table 8 



Joint Hit, Miss Rates for Moderated Keys 
Versus Other Three Predictors 



Moderated 

key 


Miss 


Hit 






General Key 




Hit 

Miss 


135 ( ^%) 
679 (20^) 


2U81 (73fO 
109 ( 3 %) 


-1.66 




Moderator 


Variable 




Hit 

Miss 


U93 (l^p) 
U98 (15^^) 


2123 (62‘/0 
290 ( 9 %) 


-7.26 




Best Combination 
Moderator 


of General Key and 
Variable 




Hit 

Miss 


137 ( 

653 (19/0 


2U79 (73%) 
135 ( ^%) 


-0.12 



test for nonindependent proportions performed according 
to McNemar (1962, pp. 52-56). 
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correctly predicted 109 of the moderated key misses. Altho^h this dif- 
ference in favor of the moderated key is statistically significant at e 

five per cent level (one-tailed test), it is certainly not great. n e 

other hand, the difference in favor of the moderated keys is substantial 
when compared with the moderator variable. Finally, it can be seen that 
there is veiy little difference between the joint hit, miss rate for the 
moderated keys and the general key in combination with the moderator 

variable. 



On the basis of these results, it can he concluded that suhsi^ary 
hypotheses SI and S2 are not supported. Thus the general hypothesis rele- 
vant to the secondary objective of the project is also not suported. 

VIhile subsidiary hypothesis S3 is supported, the overall concliKion vith 
respect to the secondary objective of the project is that, while t e 
moderated scoring key techniques can be generalized to the data un e 
investigation, their effectiveness vith this data does not warrant their 
practical application to it. In short, academic aptitude does not appear 
to be an effective moderator variable for the data under study. 

It should be pointed out that the scoring techniques developed in this 
study exist apart from the data to which they are applied. While it m^ 
be that the techniques will be of no practical value in any setting, such 
a conclusion is not warranted on the basis of the results . obtained in this 
study. It would appear that the application of moderated scoring teohniq 
to settings in which moderator variables have been known to be operating 
would be required to determine their actual value. 

Although academic aptitude was shown to be ineffective as a moderator 
variable when applied to the SIB as a whole, it may still be effective 
with a few of the items in the SIB. That is, the level of relationship 
between selected biodata items and college attendance may , itself, be 
related to the ability moderator. Figures 1, 2, and 3 are presented m 
evidence of this relationship. In each, the phi coemcients for 
item responses are shown for the twelve ability levels. Definite trends 
can be seen in each plot. It should be noted that these plots were 
selected to illustrate a point. Detailed analysis and discussion of plot 
data is beyond the scope of this report. However, the general type °f 
relationship shown in Figures 1 through 3 must be for a nmb r 

item responses in order for moderated scoring keys to be effective with 

a given set of data. 

One of the interesting aspects of the results presented in Tables 7 
and 8 is the level of accuracy achieved in the predictions. A 77> hi 
rate in the prediction of a gross and criterion 

such as college attendance is seldom found, especially with ^ h^e rate 
similar to the one in the sample under study. While the level of relation- 
ship expressed by a phi of .51* may not seem high, one must rmember that 
the phi coefficient is smaller than the Pearson product mment correlation 
coefficient in situations where it is possible to compute both (Guilford 
1956) . It is also smaller than the coefficient obtained when the point 

, . . - T . . niio TQ ii«5Pd This is apparent when the Group 4 

biserial correlation technique is usea. j-o app 

phi coefficients of .52 and .42 for the general key and moderator ^^lable 
are compared with the point biserial correlation coefficients of .58 and 
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Figure 1 



Phi Trends Across 
for Item 65, 



12 Ability Levels 
Response 2^ 




^Project item code refers to SIB item number 91, response B. 
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Figure 2 



Phi Trends Across 12 Ability Levels 
for Item 71, Response 2^ 




^Project item code refers to SIB item mamber 97, response B. 



28 



Figure 3 



Phi Trends Across 12 Ability Levels 
for Item Response 6^ 




Note. — The horizontal axis represents the 12 ability levels. 
^Project item code refers to SIB item number 122, response F. 
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,li8 obtained independently for raw scores versus criterion status. The 
phi coefficient for the general key and the moderator variable in weighted 
ccxnbination was .5^. The equivalent point biserial correlation was .60. 
These point biserial correlations also underestimate the Pearson product 
moment coefficient that would apply if the criterion (college attendance, 
or tendency toward same) could be considered to be continuous and normally 
distributed in the population rnider study. In this latter case, biserial 
correlation would be appropriate. The biserial coefficient equivalent to 
a point biserial coefficient of .60 is .75, a value which may more accu- 
rately represent the level of relationship and predictive accuracy achieved 
• in this study. 

In any case, it is interesting to note that Project TALENT (Flanagan, 
et al. , 1964) reports a multiple point biserial correlation of only .566 
between 51 cognitive predictors and college attendance for a sample highly 
similar to the one used in this study. Although the nature of the cri- 
terion differs slightly from that of the present study, there is suffi- 
cient similarity to reveal the power of biodata as a predictor. 



Conclusions 

Conclusions already stated in the section on results are simunarized 
below as they-pwrtain to the primary and secondary objectives. 

Primary Objective 

The primary objective of the project was essentially the exploration 
and refinement of techniques for developing moderated scoring keys. The 
two specific tasks which were set are separately discussed below. 

(a) Development of procedures for determi iiing the optimum number 
of subgroups (and hence, moderated scoring keys) required for maximizing 
the predictive effectiveness of an inventory. 

The procedures developed involve the formation of as many subgroups 
of equal size sis is warrsinted by the size of the available sample. It 
is considered importsuit to maintain the natural criterion distribution 
in each subgroup rather thsin achieve some arbitrsny success-failure split . 
Sets of moderated scoring keys differing widely in number per set can 
e«ily be formed from itmn ansdysis data on each subgroup and by combina- 
tion of data from pairs of subgroups. Thus sepsirate item analyses for 
each set of moderated keys is avoided. Rules were formulated for judging 
when a more general moderator key could replace a specific key. These 
rules involve the comparison of key— criterion correlations for different 
keys scored on the same group. The procedures and rules when applied to 
actual data through use of a medium-scale computer prove workable and 
economically feasible. (In fact, the total analyses for this project 
took slightly over seven hours on an IBM S60, Model liU. FORTRAN programs 
written especially for the project, were used in all analyses.) Hence, 
it is concluded that the first task under the primary objective has been 
achieved. The second task was as follows: 
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(b) Develobment of a siagle scale for icporting the scores 
obtained froea a set of moderated scoring keys , thus facilitating the 
practical application of moderated keys to prediction problems. 

In order to develop an anchor scale, it is proposed that criterion 
estimates be used as the basis for equating scores frean different 
moderated scoring keys. In this way a common scale can be formed which, 
in addition, has direct relevance to the purpose for which scores are 
obtained. The application of criterion group membership probabilities 
to the data under study proved practical and economically feasible. The 
procedure should work equally well whether criterion group membership or 
criterion scores are being predicted. In the latter case, regression 
rather than classification procedures could be used to form the common 
score scale. Hence, it is concluded that the second task under the 
primary objective has been achieved. 

Secondary Objective 

The secondary objective of the project can be described as the 
investigation of the generalizability of moderated scoring key techniques 
to a sample different fraa that on which they were first developed. The 
general hypothesis was that the contribution of biodata to the prediction 
of college attendance is greater, upon cross-validation, for scoring keys 
developed within ability level subgroups , moderated keys ) than for 

a key developed on the total group (^.£. , the general key ) . For purposes 
of evaluation, this general hypothesis was restated in the form of the 
first of the three subsidiary hypotheses listed below. 

Subsidiary hypotheses . Moderated scoring key predictions of college 
attendance-nonattendance for high school students are more accurate than 
predictions based on: (a) normal empirical keying procedures used in 

forming the general key; (b) the above, plus academic aptitude data; and, 
(c) academic aptitude data alone. Although a statistically significant 
difference (p .05) in favor of the moderated keys was obtained, the 
practical implications of this difference were slight. Hence, it is 
concluded that support for the first subsidiaiy hypothesis, and thus, the 
general hypothesis is not warranted by the data. Support for the second 
subsidiary hypotheses is also unwarranted. Predictions were as accurate 
but not more accurate. Finally, support for the third subsidiary hypo- 
thesis is warranted. The moderated scoring keys produced a substantially 
higher percentage of correct predictions . 

Byproduct of Study 

Biographical data were shown in this study to be powerful predictors 
of college attendance. The fact that a single biographical inventory 
produced correct predictions of college attendance for 77^ of the 
cases is especially surprising in view of the arbitrary way in which 
the criterion was defined. Equally surprising, in light of previous 
research, is the relative performance of biographical data and academic 
aptitude as predictors of college entrance. The predictive accuracy of 
the former substantially exceeded that of the latter. In a reverse of 
their usual roles, academic aptitude added little to the predictive 
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accuracy obtained through use of biographical data alone- Finally as 
noted in the section on results, the correlation between biographical 
data and the criterion was substantially greater than that achieved by 
means of a multiple regression equation combining variables (Flanagan, 
et al. , 196U). Hence, biographical data would appear to have considerable 
promise as predictors of who will attend college. With scmie attention 
to age'-related items, the scoring key data presented in Appendix C might 
be used to develop an inventory for this purpose. 

A final point needs to be made. Although the moderated scoring key 
techniques developed in this project were not found to produce more valid 
results than normal keying methods, the techniques themselves have not 
been shown to be without merit. Similar, though less elaborate techniques 
have been shown to work in a different setting (Prediger, 1966). It well 
may be that moderator variables are situation specific as Ghiselli and 
Sanders (1967) sxiggest. 

The procedures developed in this study could easily be applied to 
other settings, provided that a large base of data is available for 
ansilysis. It would appear that the next step would most profitably 
involve application of moderated scoring key techniques in a situation 
where moderator variables have already been shown to be operating. 
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Appendix A 

MSK.-SIB Item Equivalents^ 



MSK Ho. 


STB No. 


MSK Ho. 


SIB Ho. 


MSK Ho. 


SIB Ho. 


1 


1 


U8 


U8 


95 


121 


2 


2 


Ii 9 


h 9 


96 


122 


3 


3 


50 


50 


9 T 


123 


k 


h 


51 


51 


98 


12 li 


5 


5 


52 


52 


99 


125 


6 


6 


53 


53 


100 


126 


7 


T 


5 h 


5 h 


101 


12 T 


8 


8 


55 


55 


102 


128 


9 


9 


56 


56 


103 


129 


10 


10 


5 T 


5 T 


10 H 


130 


11 


11 


58 


58 


105 


131 


12 


12 


59 


59 


106 


132 


13 


13 


60 


60 


lOT 


133 


ih 


ih 


6l 


61 


108 


13 h 


15 


15 


62 


62 


109 


135 


l6 


l6 


63 


63 


no 


136 


IT 


IT 


eh 


eh 


111 


13 T 


l8 


18 


65 


91 


112 


138 


19 


19 


66 


92 


113 


139 


20 


20 


6t 


93 


nil 


iho 


21 


21 


68 


9I1 


115 


Ihl 


22 


22 


69 


95 


116 


lh 2 


23 


23 


TO 


96 


llT 


lh 3 


2U 


2 h 


T 1 


9 T 


118 


ll|li 


25 


25 


T 2 


98 


119 


lh 5 


26 


26 


T 3 


99 


120 


ihb 


2 T 


21 


Ih 


100 


121 


Ihl 


28 


28 


15 


101 


122 


1I18 


29 


29 


16 


102 


123 


lii 9 


30 


30 


11 


103 


12 h 


150 


31 


31 


10 


10 li 


125 


151 


32 


32 


19 


105 


126 


152 


33 


33 


80 


106 


12 T 


153 


3 h 


3 h 


81 


lOT 


128 


15^1 


35 

36 


35 

36 


82 

83 


108 

109 


129 

130 


155 

156 


3 T 


3 T 


8Ii 


110 


131 


15 T 


38 


38 


65 


111 


132 


i6t 


39 


39 


86 


112 


133 


168 


ho 


IiO 


8T 


113 


13 h 


169 


hi 


111 


88 


nil 


135 


ITO 


h 2 


h 2 


89 


115 


130 


ITl 


h 3 


h 3 


90 


116 


13 T 


1 T 2 


hh 


hh 


91 


IIT 


138 


1 T 3 


h 5 


h 5 


92 


11& 


139 


IT^ 


he 


he 


93 


119 


ihO 


?’15 


hi 


hi 


9I1 


120 


llil 


1 T 6 
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MSK Ho. SIB Ho. 



M£K Ho. SIB Ho 



MSK Ho. SIB Ho. 



lk 2 


177 


166 


256 


230 


208 


li ^3 


178 


187 


257 


231 


213 


lUij 


179 


188 


258 


232 


21I4 




180 


189 


259 


233 


215 


1U6 


181 


190 


260 


23^4 


216 


lUT 


182 


191 


261 


235 


227 


1 U 8 


183 


192 


262 


236 


218 


1U9 


18I4 


193 


263 


237 


219 


150 


185 


19^4 


26I4 


238 


220 


151 


186 


195 


265 


239 


221 


152 


187 


196 


266 


2U0 


222 


153 


188 


197 


267 


2I4I 


223 


15 ^^ 


189 


198 


268 


2U2 


226 


155 


190 


199 


269 


2I43 


230 


156 


191 


200 


270 


2I4I4 


231 


157 


192 


201 


271 


2I45 


22I4 


158 


193 


202 


272 


2I46 


225 


159 


19 


203 


273 


2I47 


228 


160 


195 


20 14 


27I4 


2I48 


229 


161 


196 


209 


275 






162 


197 


206 


276 






163 


198 


207 


277 






16 U 


199 


208 


278 






165 


200 


209 


279 






166 


201 


210 


280 






167 


202 


211 


281 






168 


203 


212 


262 






169 


20^4 


213 


283 






170 


205 


2 lh 


28I4 






171 


2^41 


215 


285 






172 


2142 


216 


286 






173 


2I43 


217 


287 






17U 


2 hh 


218 


288 






175 


2U5 


219 


2S9 






176 


2I46 


220 


290 






177 


2I47 


221 


291 






178 


2I48 


222 


292 






179 


2I49 


223 


293 






180 


250 


22I4 


29 14 






181 


251 


225 


295 






182 


252 


226 


296 






183 


253 


227 


206 






l8if 


25I4 


228 


206 






185 


255 


229 


208 







^Tho MSK Wo. rofors to th 6 itoin id 6 ntif*ica,tion nuiiilD62 usod in this 
project. The SIB Ko. refers to the equivalent Student Information 
Blank number. For copy of SIB, See Flanagan, e^. aT* , (l 9 oM. 
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Appendix E 



Changes in SIB Response Option Coding 



Project item No. 
onH T'p^^'nonsss 


SIB item No. 
and responses 


Project item No. 
and responses 


SIB item No. 
and responses 


51: 6 

1-5 


51: A 

B-F 


238 : 


1-6 

7 

8,9 


200 : A-F 

G,H 


53: 3-6 

2,1 

222 : 6 
1-5 


53: A-D 

K,F 

292 : A 

B-F 


239 , 2 U 0 
2 Ul, 2 i »2 
2h3,2hk: 


1-7 

8 

9 


221,222 

223,226 

230 , 231 : A-G 

H,I 
J-L 


227 , 229 -. 1 
2-9 

228,230: 1-8 


206 , 208 : Q 

A-H 

I-P 


21 . 5: 


1-6 

7 

8 

9 


22 U : A-F 

G,H 

K,L 


231: 1-5 

6 

7,8,9 


213: A-E 

F&I 
G,H,J 


2li6,2U7: 


1 

2-8 

9 


225 , 226 : A-C 

D-J 
K,L 


232: 1-^ 

5 

6-9 


21 U : A-D 

E,G 
F-J 


2 U 8 : 


9 

1 

2-7 

8 


229 : A 

B,C 

D-I 

J,K 


233 

23^,235: 1 

2-9 


215 

216 , 227 : A,B 

C-J 








23d&237: 1 

2-9 
0 


210&219: A,B 

C-J 
K 
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Appendix C 
Sc or inf; Keys 



Gene ral Explanat ion 

Twenty-tvo scoring keys were formed as a result of item analyses 
conducted on Group 1. Five key lengths were scored for each key. The _ 
items and responses for the general key and keys 17 and 16 are presen ..ea 
separately for responses scored +1 (indicating a tendency toward college 
attendance) and -1 (indicating a tendency toward nonattendance). Data 
for the other keys have been omitted since they were not included among 
the final scoring keys. 

In the table on the next and following pages the symbols listed 
below are used. 



K: key identification number 

IT is the key scored on ability levels 9 & 10 
l8 is the key scored on ability levels 11 & 12 
22 = general key 

L: key length index. If a negative sign appears before L, the 

responses are scored -1. If no sign appears, they are scored 

+ 1 . 



X = item responses with phi coefficients equal to or greater 

than .15. . 

2 = item responses with phi coefficients equal to or greater 

than .10, incjji^ing^ responses already listed for the key. 

3 = item responses with phi coefficients equal to or greater 

than .07, including responses already listed for the key. 



h = item responses with phi coefficient equal to or greater 
than .05, inj^ludin responses already listed for the key. 

5 = item, responses with phi coefficients equal to or greater 
than .03, responses already listed for the key. 

Items and responses are represented as one integer. The units digit 
represents the response number. All other digits represent tne item 
number. Thus, 228H would stand for project item number 228, response 4. 



39 



/ 



17 1 
17 2 


652 

11 


31 


125 


132 


301 


311 


534 


17 2 
17 3 


1522 

12 


1604 

21 


1882 

22 


2376 

23 


61 


62 


106 


17 3 


736 


786 


801 


812 


821 


823 


833 


17 3 


1385 


1394 


1415 


1425 


1474 


1482 


1575 


17 3 


2287 


2326 


2366 


2374 


2392 


2401 


2447 


17 4 


24 


33 


91 


113 


114 




126 


17 4 


342 


371 


373 


394 


405 


453 


475 


17 4 


796 


832 


871 


912 


1014 


1065 


1121 


17 4 


1204 


1224 


1231 


1273 


1302 


1354 


1366 


17 4 


1703 


1731 


2192 


2212 


2213 


2252 


2262 


17 4 
17 5 


2411 

34 


2446 

35 


2449 

46 


56 


92 


94 


95 


17 5 


355 


392 


393 


414 


431 


464 


491 


17 5 


616 


623 


625 


631 


634 


661 


67 i 


17 5 


862 


884 


896 


931 


966 


976 


1004 


17 5 


1134 


1144 


1145 


1163 


1182 


1184 


1194 


17 5 


1341 


1402 


1403 


1405 


1423 


1426 


1435 


17 5 


1503 


1505 


1523 


1526 


1533 


1542 


1545 


17 5 


1621 


1623 


1644 


1753 


1755 


1774 


1781 


17 5 


2153 


2174 


2186 


2216 


2231 


2244 


2283 


17 5 


2381 


2391 


2423 


2445 


2457 


2469 


2482 


17-1 


26 


36 


111 


536 


651 


834 


835 


17-2 


16 


66 


121 


231 


306 


376 


601 


17-2 


991 


1001 


1136 


1226 


1236 


1382 


1481 


17-2 

17-3 


1881 

96 


2361 

101 


2413 

155 


2442 

366 


2489 

395 


496 


516 


17-3 


722 


724 


732 


755 


765 


773 


774 


17-3 


962 


1061 


1081 


1102 


1166 


1196 


1206 


17-3 


1553 


1616 


1641 


1683 


1702 


1761 


1792 


17-3 

17-4 


2372 

131 


2383 

165 


2398 

185 


2399 

193 


2429 

221 


232 


254 


17-4 


545 


555 


621 


694 


705 


711 


763 


17-4 


921 


934 


963 


1011 


1072 


1095 


1113 


17-4 


1391 


1411 


1421 


1422 


1 444 


1453 


1541 


17-4 


1674 


1723 


1733 


1734 


1776 


2071 


2182 


17-4 


2319 


2348 


2396 


2403 


2405 


2417 


2428 


17-5 


42 


43 


52 


102 


103 


122 


136 


17-5 


273 


274 


304 


31 2 


314 


316 


334 


17-5 


501 


514 


525 


556 


561 


584 


586 


17-5 


675 


692 


693 


721 


723 


734 


741 


17-5 


855 


866 


933 


935 


944 


956 


961 


17-5 


1093 


1126 


1153 


1176 


1185 


1186 


1195 


17-5 


1352 


1376 


1412 


1433 


1443 


1445 


1 456 


17-5 


1634 


1642 


1666 


1684 


1694 


1735 


1751 


17-5 


2191 


2195 


2201 


2223 


2241 


2264 


2272 


17 -5 


2332 


2337 


2342 


2346 


2347 


2351 


2352 


17-5 


2402 


2404 


2408 


2414 


2415 


2426 


2427 



702 


771 


811 


831 


843 


873 


886 


996 


112 


234 


235 


271 


302 


615 


715 


733 


841 


842 


872 


926 


1006 


1074 


1232 


1375 


1585 


1596 


1791 


1801 


1981 


1991 


2224 


2286 


2448 


2483 














151 


162 


164 


205 


215 


225 


263 


281 


502 


511 


572 


606 


714 


716 


726 


756 


1133 


1156 


1161 


1162 


1173 


1192 


1193 


1202 


1384 


1451 


1473 


1484 


1506 


1556 


1645 


1662 


2284 


2307 


2315 


2335 


2344 


2357 


2367 


2375 


104 


141 


152 


153 


173 


252 


294 


321 


492 


493 


522 


531 


552 


583 


603 


604 


701 


725 


735 


764 


813 


822 


851 


861 


1015 


1026 


1063 


1064 


1085 


1091 


1112 


1131 


1201 


1222 


1263 


1284 


1285 


1303 


1305 


1333 


1436 


1441 


1475 


1483 


1486 


1492 


1495 


1496 


1565 


1576 


1583 


1584 


1595 


1612 


1613 


1615 


1872 


1941 


1971 


2011 


2021 


2102 


2142 


2143 


2285 


2292 


2304 


2328 


2353 


2365 


2368 


2369 


845 


874 


881 


922 


2441 








654 


704 


712 


731 


815 


844 


854 


875 


1521 


1531 


1571 


1572 


1581 


1602 


1663 


1 664 


521 


544 


576 


611 


636 


664 


706 


713 


775 


791 


805 


814 


826 


846 


882 


911 


1306 


1361 


1381 


1392 


1404 


1471 


1491 


1501 


1802 


1982 


1992 


2275 


2279 


2281 


2298 


2371 


262 


292 


313 


336 


346 


375 


402 


471 


776 


782 


783 


836 


864 


865 


893 


895 


1146 


1276 


1286 


1336 


1346 


1353 


1371 


1372 


1551 


1552 


1564 


1582 


1591 


1601 


1626 


1673 


2211 


2234 


2251 


2271 


2274 


2299 


2303 


2308 


2431 


2452 














143 


171 


172 


201 


212 


222 


223 


233 


343 


353 


382 


401 


415 


455 


462 


481 


591 


624 


644 


655 


665 


666 


672 


674 


743 


751 


752 


781 


784 


785 


816 


825 


971 


972 


992 


1021 


1023 


1024 


1042 


1075 


1214 


1215 


1266 


1296 


1314 


1325 


1343 


1344 


1465 


1472 


1561 


1563 


1592 


1594 


1624 


1625 


1772 


1782 


1871 


1942 


2051 


2061 


2101 


2141 


2277 


2278 


2293 


2296 


2314 


2321 


2325 


2329 


2358 


2362 


2363 


2373 


2382 


2387 


2394 


2397 


2464 


2479 


2486 













ko 



K L 



ITEMS 



A i» D 



RESPONSES 



18 


2 


405 


811 


831 


966 


976 


18 


3 


21 


61 


235 


311 


475 


18 


3 


861 


871 


872 


1006 


1026 


18 


3 


2449 










18 


4 


31 


34 


56 


92 


106 


18 


4 


465 


474 


482 


491 


534 


18 


4 


911 


926 


931 


996 


1132 


18 


4 


1482 


1522 


1556 


1575 


1586 


18 


4 2366 


2376 


2377 


2448 


2468 


18 


5 


11 


12 


13 


14 


22 


18 


5 


125 


126 


144 


145 


152 


18 


5 


294 


301 


302 


334 


335 


18 


5 


552 


582 


592 


596 


602 


18 


5 


823 


832 


842 


843 


892 


18 


5 


1121 


1122 


1134 


1144 


1162 


18 


5 


1 244 


1291 


1292 


1293 


1303 


18 


5 


1493 


1503 


1523 


1524 


1525 


18 


5 


1711 


1715 


1753 


1764 


1872 


18 


5 


2176 


2192 


2196 


2205 


2231 


18 


5 


2423 


2438 


2439 


2445 


2451 


18 


-1 


834 


874 









18-2 
lG-2 
18-3 
18-3 
10-3 
18-3 
18-3 
18-A 
1 0 -4 
13-4 
IP -4 
If- -4 
18-4 
1 r -5 
18-5 
1 G -r 
If 

r: 

i: 
r--5 
10 -5 
10-5 
1 0 -5 
1G--5 



.r. 

j - 

-C 



26 
1136 
16 
526 
864 
1236 
2221 
56 
653 
1034 
1412 
1 642 
2318 
33 
2C 3 
61 
782 
1 O-'V: 
1345 
1-481 
1712 
lSo2 
2333 
2477 



121 
1382 
36 
654 
855 
1 276 
2236 
I4l 
673 
1 042 
1 41 4 



2325 

h 0 

0 O') 

1 •? .C 






"> 



1011 

1353 

1 486 

1724 

1552 

2333 

2475 



376 
1521 
66 
656 
S’22 
1 256 
2241 
143 
632 
1 043 
1442 
1674 
233 s 
53 
316 
641 
836 
1 0.:1 
1 356 
1 45 1 
1744 
20-':1 
2347 
2487 



471 
1663 
102 
666 
961 
1306 
2251 
151 
703 
1071 
1 443 
1732 
2342 
55 
322 
655 
o46 
1031 
1383 
1456 
1755 
2081 
2332 



2141 
111 
672 
962 
1352 
2324 
221 
721 
1096 
1 453 
1735 
2348 
75 
3 45 

i:'o r 

854 
1 053 
1 406 
1515 
1775 
2122 
2335 



522 652 716 
224 1273 1375 

133 175 

571 584 

1201 1232 
2162 2204 



115 

541 

1156 

1731 

2483 

24 

161 

355 

642 

923 

1163 

1304 

1533 

1951 

2270 

2453 

732 
2361 
171 
712 
972 
1404 
2331 
253 
723 
1105 
1 -6-56 
1871 
2 3 S' 4 

o2 
3S'Z 
67 4 
< 55 
1081 
1 6-1 1 
1545 
1 782 
221 1 
2403 



32 

174 

356 

66l 

941 

1171 

1305 

1542 

2042 

2302 

2462 

804 
2337 
21 1 
713 
973 
1 462 
2341 
262 
705 
1155 
1531 
21^:1 
2408 
51 
i^02 
6(>4 
865 
1113 
1 41 3 
1552 
1752 
2222 
2404 



46 

205 

372 

671 

955 

1173 

1341 

1565 

2052 

2315 

2473 

805 
2441 
231 
731 
1001 
1 471 
2362 
401 
335 
1176 
1541 
2201 
241 8- 
105 
403 
651 
^16 
1 1 

1 422 
1562 
1802 
2226 
2412 



733 

1384 

195 

635 

1385 
2224 

86 

212 

393 

683 

986 

1174 

1363 

1584 

2082 

2316 

2484 



771 801 802 821 822 841 

1415 1451 2143 2287 2369 2446 

225 255 362 371 435 445 

715 786 794 796 812 886 

1401 1416 1 424 1 441 1461 1475 

2245 2252 2284 2326 2334 2344 



93 
215 
394 
702 
1036 
1184 
1 364 
1612 
2142 
2317 



8 1 



306 

755 

1023 

1501 

2371 

434 

o62 

1225 

1553 

2275 



202 

423 

705 

r 7 b. 

1 246 
1431 
15S-1 
1G12 
2232 
2415 



314 
765 
1025 
1603 
2372 
504 
875 
1255 
1564 
22 7 S 
24b4 
213 
483 
706 
936 
1 266 
143 '8 
1593 
IC'32 
2233 
2416 



94 

222 

425 

704 

1041 

1193 

1394 

1621 

2144 

2328 



112 
245 
472 
726 
1062 
1 202 
1405 
1644 
2152 
2335 



113 

265 

492 

754 

1103 

1221 

1434 

1645 

2153 

2367 



114 

271 

512 

756 

1104 

1222 

1473 

1662 

2163 

2381 



124 
275 
513 
761 
nil 
1234 
1 492 
1703 
2172 
9 '^C ? 



816 025 826 b44 845 1061 



431 

773 

1102 

1641 

2398 

543 

881 

1336 

1571 

22P1 

6 

224 
515 - 
711 
952 
1 2^^ 
1 i-M 
1 602 
18-52 
2271 
2425 



8'41 

783 

1126 

1673 

2399 

576 

956 

1361 

1572 

22 sr 

9/.pr 

233 

521 

7f’1 

CJ Li. 

1312 
1 i'£;9 
1 61 6 
1861 
2295 
244'! 



454 

791 

1154 

2151 

2442 

601 

963 

1362 

1581 

2299 

251 
^■46 
753 
: < 1 
1321 
1 -^'54 
1651 
18G1 
2312 
2452 



496 
815 
1 1 66 
2161 



611 

965 

1371 

1601 

230-'- 

254 

l \ k - 

1322 
1 i-63 
] CC .: 
1892 
2322 
2456 



C . 2 S 

■ E -3 

1226 

2171 

625 
SSI 
1 ^72 

U-34 
231 4 

:75 

5^4 

772 

( c '• 
1 - 2 
1 

1 7 0 : 
1 ‘‘ 02 
233 
9i 72 



kl 




22 2 



22 

22 

22 

22 

22 

22 

22 

22 

22 

22 

22 

22 

99 



2 
2 
2 
3 
3 
3 
3 
3 

3 

4 
4 
4 
4 

22 4 
22 



225 
1565 
31 
oOl 
1335 
2366 
1 1 
301 
745 



652 
1 604 
56 
821 
1403 
2365 
1 2 
302 
802 

1256 1351 

1832 1851 
2252 2283 
24 46 

641 651 

1036 1064 1103 
1374 1401 1414 
1644 1661 1672 
4 2162 2196 2212 
4 2423 2433 2 -'■•67 



726 
2287 
61 
822 
1415 
2375 
21 
303 
812 
1366 
1882 
2234 
86 
746 



215 

823 
1441 
2376 
22 
342 
813 
1384 
1532 
2285 
91 
762 
1121 
1 41 6 
1704 

2216 

2465 



22 


5 


33 


62 


71 


22 


5 


21 4 


245 


263 


22 


5 


59-4 


602 


604 



11 5 
11 5 
11 5 
22 5 
22 5 
22-1 
22-1 
22 -2 
22-2 
22-2 
22 -'’ 
22-3 
22 -3 
22-3 
22-3 
22-3 
22 -3 
22-4 
22-4 
22 -4 
22-4 
22-4 
22-4 
22-5 
22 -5 
22-5 
22-5 
22-5 
22-5 
22-5 
22-5 
22-5 
22-5 
22-5 
22-5 



1014 1015 
1234 1277 1273 

1503 1523 
IG5I 
2245 
121 
881 
36 
742 
561 

1552 
53 
595 
306 
1372 

1 641 
2'f63 
75 
654 
1 1 56 
1561 
2041 
2^-! 04 
42 

153 

422 

545 

853 
1 036 
1314 

151 2 

1735 
21 21 
2331 

2415 



1475 
1872 
2223 
26 
875 
16 
721 
521 
1521 
52 

504 
791 
1371 
1631 
2431 
54 
653 
1 043 
1553 
2031 
2398 
35 
152 
403 
544 

844 

1 081 

1312 

1 454 

1724 
21 1 1 
2324 
241 2 



1922 

2261 

306 

522 
66 
753 
562 
1562 
96 
656 
816 
1301 
1 665 

2485 
252 
713 
1206 
1582 
221 1 
2405 

51 

213 

423 

546 

856 

1115 

1 326 
1513 
1761 
2165 
2332 

2416 



53 
26 '-f 
61 4 
1062 
1251 
1555 

1Q41 

2320 

601 

551 

111 

755 

572 

1563 

155 

704 

836 

1382 

1673 



235 

833 

1461 

2352 

23 

371 

832 

1354 

1562 

2286 

114 
756 

1131 

1423 

1712 

2224 

115 

255 
662 
1 0;'5 
1 252 
1 5^ 6 
1571 
2334 
61 1 
1 001 
221 
763 
573 
15^ 1 

165 

705 

864 
1391 
1 674 



271 

841 

1522 

2401 

32 

425 

861 

1473 

1981 

2315 

123 

803 

1132 

1424 

1741 

2244 

144 
442 
676 
1 1 04 

1293 
1 594 
2032 
2343 
651 
1 061 
111 
773 
1075 
1601 
21 1 
706 
874 
1 406 
1683 



311 
842 
1556 
241 1 
<^2 
475 

884 

1502 

2022 

2367 

132 

824 

1134 

1425 

1774 

2288 

151 
451 
701 
1 1 22 
1301 
1596 
2042 

2351 

654 

1136 

231 

776 

1105 

1663 
21 2 
711 
882 
1421 
1776 



405 
843 
1 1'45 
2446 
106 
491 
891 
1533 
2102 
2374 
234 
871 
1 161 
1451 
1812 
2302 

152 

465 

765 

1133 

1302 

1615 
2062 
2393 
712 
1471 
316 
783 
1 1 26 
1 664 
223 
7^il 
963 
1443 

2271 



616 

872 

1662 

2448 

112 

501 

925 

1575 
21 12 
2381 
321 
885 
1162 
1474 
lc .22 

2317 

153 

iiC 9 

792 
1 1 44 
1303 

I 633 
2132 
2424 

722 

1551 

376 

805 

I I 66 
1702 

232 

752 

971 

1463 

2272 



715 

o73 

1731 
2449 
113 
522 
986 
1584 
21 22 
2439 
553 
906 
1163 
1482 
1841 
2335 

162 
511 
794 
1182 
1365 
1671 
21 61 
2437 
731 
1602 

536 

826 

1236 

2275 

201 

754 

1011 

1501 

227? 



253 
723 
1253 

1553 

2251 

2-'il3 

55 

254 

434 

576 

876 

1146 

1344 

1516 

1771 

2182 

2338 

2421 



275 

743 

1352 

1642 
2281 
241 4 

03 

262 

444 

892 

1151 

1345 

1573 

1782 

2201 

2335 

2426 



305 

751 

1353 

1646 

2291 

2428 

84 

265 

454 

621 

903 

1152 

1346 

1592 

1802 

2221 

2347 

2427 



366 
81 4 
1362 
1 666 
231 4 
2425 
c5 
273 
462 
644 
934 
1153 
1356 
1616 
1821 
2233 
2349 
2432 



424 

l'6b 

1376 

1675 

2341 

2643 

101 

274 

471 

666 

942 

1154 

1413 

1636 

1842 

2234 

2351 

2461 



524 526 

893 933 

1386 1442 
1676 1733 
2342 2383 
9451 2462 



716 

1006 

19S1 

1 24 
534 
1 041 
1595 
2142 
2445 
596 
924 
1 221 
151 1 
1901 
2344 

163 
512 
-46 
1193 
1393 
1682 
21 73 
2454 
781 
2361 
556 

1361 

2319 

314 

764 

1021 

1551 

2299 

543 

974 



102 

313 

503 

673 

964 



1 03 

315 

505 

674 

975 



1155 1251 

1436 1 445 
1653 1654 
1881 1931 
2235 2236 

2355 2382 
2471 2479 



1734 

2387 

2464 

105 

336 

513 

692 

932 

1252 

1455 

1655 

1961 

2273 

2385 

2486 



725 

1063 

2231 
125 

541 

1065 

1703 

2152 

2447 

603 

931 

1231 

1532 

1911 

2357 

164 

563 

951 

119^ 

1402 

1722 

9.192 

2457 

762 

2641 

561 

8^6 

1392 

2371 

346 

765 

1023 

1564 

2325 

555 

1022 

1446 

1792 

2394 

2488 



fal 



735 736 . 

1074 1156 1375 
2307 2326 2365 



126 

606 

1073 

178.1 

2172 

2 iS 3 

613 

995 

1316 

1542 

2002 

2364 



255 
702 
1135 
1791 
2181 
2484 
615 
1 004 



294 

714 
1 232 
1801 
2206 



631 
1 005 

1 341 
1576 1 585 
2012 2143 

2368 2377 



1333 



174 

565 

CCh 



195 
592 
101 2 
1201 1202 
1405 1432 
1763 1773 
2202 2213 
2468 
835 



015 

qc-l 

8.54 

1 /.^04 
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Appendix D 



Item Two of Project TALEKT 19^>1 Follow-up Questionnaire 



Have you attended college since leaving high school? 

1. Yes, as a full-time student. 

2. Yes, as a part-time student. 

3. Yes, I entered but have dropped out temporarily. 

li. Yes, I entered but dropped and do not plan to return. 

5 :-;o, but I plan to enter college within a year or two. 

6. lio, but I plan to enter college eventually; I have no 

idea when. 

T. Ho, and I have no plans to do so. 
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i The three major project objectives were as follows: 
i (l) Development of procedures for determining the optimum 
I number of subgroups (and hence, moderated scoring keys) 
required for maximizing the predictive effectiveness of an 
inventory; ( 2 ) Development of a single scale for reporting 
the scores obtained from a set of moderated keys, and, ii) 
Determination of the accuracy of moderated scoring key pre- 
dictions of college attendance as compared with predictions 
obtained from conventional keying techniques. 

Basic data consisted of biographical inventory responses 
and academic aptitude test scores for approximately 20,500 
high school senior boys . Scoring keys were formed _ for each 
of 12 ability level subgroups and various combinations oi 
these subgroups. The keying procedures developed as the pri 
mary objective of the project work well when applied to 

actual data. . 

a Rtatisticallv significant difference xn favor 

" . - • ^ 14 4 - 1 *- T.roc 
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Moderated scoring keys, inventory keying, empirical keying, . 
scoring keys, moderator variables, item analysis, tesT ; 
scor'ng, biographical data, prediction of college attendance i 
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Although 



of the" moderated keys was obtained, academic ability was not 
f-.und to be an effective moderator variable.^ However, a i 
rate of 77 per cent was achieved by biographical data as a 
preaictor'of college attendance versus nonattendance . ^is 
kte and the equivalent point biserlal correlation coefficient 
of .60 were substantially higher than the correspondi>,g iigures 
for academic aptitude used alone. 
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